1. Backgroundxe2x80x94Field of the Invention
This invention relates to a novel high-resolution color video camera having high-resolution color image capture and, more particularly, a high-resolution color video camera having two different image sensors and a signal processing apparatus for generating a high-resolution color image.
2. Backgroundxe2x80x94Description of the Prior Art
Color video cameras are well known and used for many years in television production, news gathering and cinematography. Recent improvement in the sensors manufacturing process made possible the development of high-resolution color sensors with several million active pixels, and thus the development of high-resolution (high definition) video cameras, which are to replace the conventional standard definition television cameras and to enable the use of these new cameras in the motion picture industry. Parallel with that, new high definition video standards have been developed to facilitate this process.
Several types of solid-state video cameras utilizing a plurality of image sensors are already known. Prior art cameras, such as described in the U.S. Pat. Nos. 4,823,186 and 4,876,591 to Muramatsu, use a beam splitter and two image sensorsxe2x80x94one monochrome and one color. The obtained video signal is a combination of two luminance signals, YL and YH, and two low frequency color difference signals, RL-YL and BL-YL. The monochrome imager generates only the high-resolution portion YH of the luminance signal Y. The color imager generates the low-resolution portion YL of the luminance signal and the two color difference signals RL-YL and BL-YL. The camera device is intended for an interlaced NTSC application, in which a video frame is captured every {fraction (1/30)} of a second ({fraction (1/60)} of a second for odd fields and {fraction (1/60)} of a second for even fields). In this mode of interlaced scanning an artifact, known as interline flicker, is present. This artifact is due to xe2x80x9cMoire patternsxe2x80x9dxe2x80x94high-resolution information near the Nyquist limit of resolution that produces a beat at low spatial frequencies resulting in flicker at 30 Hz (for NTSC standard signals). To avoid xe2x80x98Moire patternxe2x80x99, Muramatsu advised us that the bandwidth for YL, RL-YL and BL-YL signals should be from 0 to 0.7 MHz, and the bandwidth for YH should be from 0.7 to 4.2 MHz. Unfortunately, these bandwidth restrictions limit the application of the camera device to the conventional NTSC standard. Being limited to NTSC standard, which is interlaced in its nature and has a bandwidth limit of 4.2 MHz, the method is not applicable to high-resolution machine vision and computer cameras, which predominantly use progressive scanning to eliminate xe2x80x98Moire patternxe2x80x99. Furthermore, this camera is not compatible with newly developed digital and high definition television (HDTV) standards, which also use progressive scanning with 480 lines per frame, 720 lines per frame, and 1080 lines per frame. In addition, high definition luminance Y and the corresponding color difference signals CR and CB have bandwidth much higher than NTSC. For example, for 720 lines per frame, the luminance bandwidth is about 30 MHz, and CR and CB bandwidth is 15 MHz each. Moreover, the low frequency luminance signal YL is derived from the color imager, so an imager with tri-color filters (red, green, blue or yellow, cyan, magenta) is required. Thus, the implementation of an imager with only two types of color filters (such as red and blue) is not possible.
In U.S. Pat. No. 5,978,023 to Glenn, and in the article xe2x80x98The Development of a 1920xc3x971080 Pixel Color CCD Camera Progressively Scanned at 60 Frames per Secondxe2x80x99, by Glenn and Marcenka, a video camera is described, where together with line and pixel interpolation the camera uses high-pass filtering and frame interpolation on the luminance channel for increasing the frame rate. As a result a high-resolution 1920xc3x971080 image scanned at 60 frames per second is generated. The output data rate of this image is about 3.3 Giga bits per second, which is beyond the capacity of the current analog or digital data recorders, editing and transmitting systems. In the majority of applications this increasing of the frame rate is not justified, does not improve the picture quality, and unnecessarily overloads and increases the computational time. In addition, particularly for digital cinema applications image scanning is only at 24 frames per second. In some special effects fast motion capture is necessary, but 60 frames per second is not fast enough and further complicates the recording, because the existing video recorders can record either normal speed at 24 frames per second or fast capture at three times the speed, i.e. 72 frames per second. In addition, this fast capture mode is used for selected frames only, and not for the entire production.
Accordingly, it is a general object of this invention to provide a novel apparatus for high-resolution and high frame rate image capture that can operate at the newly developed high definition television and digital cinema standards. It is a further object of the present invention to provide an improved progressively scanned high-resolution color camera apparatus, which is compatible with the existing data recording and editing equipment. It is also a further object of tile present invention to provide an improved progressively scanned high-resolution color camera apparatus and method for machine vision and computer applications.
These and other objects, advantages and features of the present invention will become more obvious from the following detailed description of the preferred embodiments along with the accompanying drawings.
Accordingly, the present invention provides a novel high-resolution video camera able to capture high-resolution images with high frame rate, where the video signal is produced using one monochrome and one color image sensors, and a signal processing apparatus. This novel apparatus does not involve frame rate temporal interpolation and high-pass filtering in its work.
According to one preferred embodiment of the present invention, the high-resolution camera includes a camera lens, a beam splitter, and two image sensorsxe2x80x94one color and one monochrome, optically aligned with the lens and the beam splitter. The optical beam received from the camera lens is equally split into two beams via a 50/50 beam splitter and each beam projects an image onto the corresponding image sensor. The beam splitter can have unequal dividing ratio, such as 60/40, 30/70 or others. Both image sensors have the same active area and the same number of pixels horizontally and vertically. If sensors with different active areas are used, an additional lens or set of lenses is used to alter one or both optical beams divergence angle, modifying the optical beam cross-section in such a way that the new image size corresponds to the image sensor""s active area. Thus, regardless of their size, both sensors will capture exactly the same scene.
In this embodiment both image sensors are scanned synchronously using a progressive scanning mode. In this particular application the monochrome imager has 1920 (MH=1920) horizontal pixels per line and 1080 (Mv=1080) vertical lines. This imager is scanned at 24 frames per second. The output signal is digitized with 12-bit A/D and results in a high-resolution luminance signal YH.The resolution of this signal is 1920 (YHH=1920) horizontal pixels per line and 1080 (YHV=1080) vertical lines. An additional 12-to-10 bit gamma correction can be used.
The color imager has red, green and blue color filters arranged as RGBG vertical stripes, i.e. the first stripe has red color, the secondxe2x80x94green, the thirdxe2x80x94blue, the fourthxe2x80x94green, the fifth is again red, an so on. This imager is scanned at 72 frames per second adding three lines at a time (3:1 binning), and generates three low-resolution color signals red, green and blue. The spatial resolution of the red and blue signals is 480 horizontal pixels per line and 360 vertical lines, and the spatial resolution of the green signal is 960 horizontal pixels per line and 360 vertical lines. These signals are digitized using a triple 12-bit A/D converter, and then converted to two low-resolution color difference signals, CR and CB, and one low-resolution luminance signal YL. The spatial resolution of each color difference signal CR or CB in this particular application is 480 (CH=480) horizontal pixels per line and 360 (CV=360) vertical lines. The spatial resolution of the low-resolution luminance signal YL is 960 (YLH=960) horizontal pixels per line and 360 (YLV=360) vertical lines. An additional 12-to-10 bit gamma correction can be used on each signal.
Both image sensors are scanned synchronously as described below. At frame 1 both image sensors are scanned, at frames 2 and 3 only the color sensor is scanned, at frame 4 both image sensors are scanned, at frames 5 and 6 only the color sensor is scanned, at frame 7 both image sensors are scanned, and so on. In this embodiment YH1, YH4, YH7 and so on, represent the high-resolution luminance from frames 1,4,7 and so on, and CR1, CB1, YL1, CR2, CB2, YL2, CR3, CB3, YL3, and so on, represent the low-resolution luminance and color differences from frames 1,2,3 and so on. Thus a synchronous scanning at 24 frames per second for the monochrome imager and 72 frames per second for the color imager is achieved. In this mode of scanning each frame from the monochrome imager has a corresponding color frame.
The signals CR, CB and YL are supplied to a signal processing module. First, the signals CR1 and CB1 are digitally interpolated 1:4 in horizontal direction and 1:3 in vertical direction to CHR1 and CHB1, which have the same pixel resolution as the corresponding high-resolution luminance signals YH1. CHR1, CHB1 and YH1 are temporally aligned using an appropriate delay line and digitally mixed together. The same procedure is repeated for the frames 4, 7, etc. In this way a high-resolution image scanned at 24 frames per second is generated. On the next step the low-resolution luminance signals YL1, YL2 and YL3, are selected, and two new motion vector signals MY2 and MY3 are generated using frame subtraction. MY2 is generated when YL1 is subtracted pixel by pixel from YL2, and MY3 is generated when YL1 is subtracted pixel by pixel from YL3. The same procedure is repeated for frames 4, 5, 6, for frames 7, 8, 9, and so on. The existing editing equipment can use this motion information to generate special motion effects, which will be added to the existing high-resolution image. For more precise motion an additional set of motion signals MR2, MR3 and MB2, MB3 can be derived in the same way from CR1, CR2, CR3 and CB1, CB2, CB3.
The newly generated motion signals MY2, MY3, MR2, MR3, MB2, and MB3 are mixed together with the newly generated high-resolution image forming in this way a composite data stream containing all image and motion information. Additional information such as date, time, camera location, etc. can be added to the data stream if needed. This composite data stream is further compressed if required to a 1.5 Giga bits per second data rate, and thus it can be recorded on the existing data recorders.
In some specific applications high frame rate information is not required, so both imagers can be scanned with the same frame rate.